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Abstract 


!  Continuous  time  Markov  chains  are  commonly  used  in  system  reliability  modeling. 
Increasing  system [complexity  and  non-Markovian  behavior  can  drastically  increase 
the  size  of  a  Markov  model’s  state  space.  Special  approximation  techniques  and 
numerical  methodb  have  been  introduced  to  reduce  the  resources  needed  to  solve 
Markov  chain  models.  ^In  ' this  paper  discussers  method  for  automatically 
deriving  exact  transient  solutions  of  Markov  chains.  The  solutions  derived  are 
symbolic  in  t  <  approach  can  also  provide  solutions  that  are  symbolic  in  other 
parameters.  We  extend  our  method  to  include  parametric  sensitivity  analysis  of 
the  transient  solution,  and  to  provide  cumulative  measures  of  Markov  chain 
— — behavior.  We  present  three  examples,  one  to  show  the  use  of  our  method  in 
evaluating  approximate  solution  techniques,  one  showing  parametric  sensitivity 
analysis  of  a  large  Markov  model,  and  one  demonstrating  the  computation  of 
cumulative  measures  for  an  acyclic  Markov  reward  processes. 
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1.  Introduction 

Continuous  time  Markov  chains  (CTMC)  are  commonly  used  tools  in  computer  systems  modeling. 
CTMC  have  been  used  to  model  program  behavior,1  system  performance,2’ 3  system  reliability,4’5’6  and 
system  availability,7’ 8  and  also  in  the  combined  evaluation  of  performance  and  reliability.9’ 10’ 11  Although 
the  limitation  of  exponentially  distributed  state  occupancy  times,  as  implied  by  a  homogeneous  CTMC, 
appears  to  be  restrictive,  it  is  possible  to  use  the  Coxian  method  of  stages  to  allow  arbitrary  phase  type 
distributions.  3> 6- 12> 13 

Much  work  done  on  Markov  chain  solution  techniques  has  been  motivated  by  queueing  theory. 
Consequently,  these  efforts  have  concentrated  on  finding  the  "steady-state”  solution  of  a  Markov  chain. 
In  contrast,  reliability  modeling,  for  example,  is  interested  primarily  in  transient  solutions  of  Markov 
chains.  Once  a  Markov  chain  model  of  a  system  has  been  constructed,  several  methods  for  finding  its 
transient  solution  are  available.  Table  1  lists  Markov  chain  transient  solution  methods  and  reliability 
modeling  packages  that  employ  them.  A  more  detailed  discussion  of  these  reliability  modeling  packages 
can  be  found  in  the  literature.5  We  briefly  summarize  the  various  methods  here  and  provide  appropriate 
references. 

The  Markov  model  of  a  system  can  be  solved  using  integral  equations,  formally  taking  the 
convolution  of  the  probability  of  entering  the  state  with  the  probability  of  remaining  in  it.14>  15 
Alternatively,  the  Markov  chain  can  be  converted  to  a  coupled  set  of  homogeneous  differential  equations.3 
This  set  of  equations  can  be  solved  using  either  numerical  techniques  or  Laplace  transforms.  The  third 
fundamental  approach  to  transient  solution  is  to  convert  the  transition  rate  matrix  to  a  form  from  which 
the  solution  can  be  easily  obtained.  Uniformization  (or  Randomization)  is  a  transformation  method  that 
solves  a  CTMC  by  using  a  related  discrete  time  Markov  chain  (DTMC).14, 18  Other  solution  methods 
based  on  these  three  basic  approaches  include  approximation  techniques  for  evaluating  large  models,17, 18 
and  the  closed  form  exact  technique  that  we  present  in  this  paper.  Before  we  discuss  our  closed  form 
solution  technique,  we  will  give  an  example  of  a  typical  acyclic  Markov  model. 

Our  example  comes  from  a  reliability  model  of  an  avionics  computer  system.19  Figure  1  depicts  the 
transition  diagram  of  a  Markov  chain  representing  a  TMR  parallel  redundant  system  with  two  cold 


spares.  The  individual  components  have  lifetimes  that  are  independent  and  exponentially  distributed  with 
parameter  X.  The  cold  spares  are  assumed  not  to  fail.  When  an  individual  component  fails,  a 
reconfiguration  process  with  rate  parameter  6  begins.  This  process  is  guaranteed  to  reconfigure  the  system 
as  lcr'.g  as  a  second  fault  does  not  occur  before  the  reconfiguration  is  completed.  The  reliability  of  the 
system  at  time  t,  denoted  R(t),  is  l-P  {process  is  in  a  non-functioning  state  at  time  <}.  So  for  the 
system  in  Figure  1,  R  (t )  is  given  by  1  -  P  {process  in  state  A  at  time  t  }. 

We  will  present  a  method  for  the  automatic  derivation  of  state  probabilities  of  an  acyclic  Markov 
chain  using  closed  form  expressions  that  are  symbolic  in  t .  This  approach  can  be  extended  to  solve  the 
CTMC  symbolically  in  terms  of  other  parameters.  Although  this  computation  might  be  done  using  an 
existing  symbolic  mathematics  package,  a  package  developed  specifically  for  Markov  chain  solution  is 
simpler,  faster,  and  can  be  easily  interfaced  with  existing  performance  modeling  tools.  Exact  symbolic 
results  (that  previously  could  be  obtained  only  by  hand)  give  greater  insight  into  actual  system  behavior 
by  allowing  us  to  easily  study  the  relationship  between  input  parameters  and  the  resulting  state 
probability  distributions.  Exact  symbolic  solutions  also  allow  us  to  use  Markov  models  as  the  basis  for 
design  optimization.20  The  derivation  of  our  approach  is  based  on  the  use  of  integral  equations.  In  some 
ways,  it  is  computationally  similar  to  using  symbolic  Laplace  transforms  to  solve  systems  of  coupled 
ODE’s.  The  program  implementing  our  algorithm  is  called  ACE  (Acyclic  Markov  Qhain  Evaluator). 
We  omit  the  discussion  of  the  solution  of  cyclic  chains  in  this  paper.  The  analysis  of  a  n  -state  single¬ 
entry  single  exit  cycle  (a  simple  case)  requires  the  computation  of  the  complex  roots  of  a  polynomial  of 
degree  n  +1.  In  general,  this  is  computationally  expensive  and  probably  not  feasible  for  purely  symbolic 
solution. 

In  section  2,  we  describe  the  mathematical  basis  for  our  method  that  is  partially  inspired  by  the 
program  SPADE.21  We  then  discuss  the  computation  of  "instantaneous  measures"  of  CTMC  behavior  and 
describe  the  implementation  of  our  procedure.  In  section  3,  we  discuss  parametric  sensitivity  analysis  of 
instantaneous  measures.  "Cumulative  measures”  of  CTMC  behavior  are  outlined  in  section  4.  Three 
acyclic  CTMC  are  analyzed  in  section  5,  demonstrating  the  use  of  the  various  measures  and 
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computational  techniques  previously  described. 

2.  State  Probability  Computation  and  Instantaneous  Measures 


2.1  Basic  Approach 

Consider  an  acyclic  continuous  time  Markov  Chain.  Let  the  states  be  numbered  1,2,...,N. 

Q  =  {9,-/}  is  the  infinitesimal  generator  matrix  of  the  CTMC.  The  transition  rate  from  state  1  to  state 

j  is  denoted  by  9,y,  and  is  restricted  to  be  non-negative.  We  let  q{  =£] qi}-  denote  the  total  exit  rate 

i 

from  state  t . 

For  any  state  «  of  an  acyclic  Markov  chain,  let  P,  (f )  be  the  probability  that  the  system  is  in  state 
»  at  time  t .  For  any  state  « ,  P,-  (t )  may  be  written  as  a  polynomial  of  the  form 

Pd 0  =  E«1<,‘  [e  *a 

i  * 

The  fact  that  the  state  probability  distributions  are  of  this  form  is  easily  derived.  First  observe  that  the 
initial  state  probabilities  have  this  form.  If  we  condition  on  the  time  of  transition  into  (any  non-initial) 
state  i ,  the  probability  of  being  in  that  state  at  time  t  can  be  written  as: 


P<(*)  =  /  E  P,(z)?*e~,,('”)rf*  (2) 

0  /  eJ(>  ) 

where  J(i)  is  the  set  of  states  with  a  transition  leading  into  state  ».  By  induction,  it  is  easy  to  show 
that,  if  every  Py  (x )  has  the  form  (1),  then  so  does  any  P,-  (x )  derived  using  (2). 

We  now  derive  the  equations  needed  to  calculate  the  constants  of  equation  (1)  for  any  state  of  an 
acyclic  Markov  chain.  Let  r(«)  be  the  set  of  poles  of  the  Laplace-Stieltjes  Transform  of  P,-(t),  i.e.  the  set 
of  the  7,-y 1  a  of  (1).  We  rename  7'  -  9,  and  define  r(/(«))  M  r(j).  Setting 

L(j)  =  |  r(y )  j  ,  we  can  write  T(y)  =  {7/i,7y2>  •  •  •  ,  7 ;£(/)}•  With  this  numbering  of  the  poles,  P;(t ) 
may  be  written  as: 


(3) 


where  K(j,l)  is  the  maximum  power  of  t  associated  with  pole  7 y,  in  Py(t)  such  that  the  formal 
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expression  of  a,u,  is  different  from  zero.  If  L  ( j )  >  1,  it  is  easy  to  show  using  an  inductive  proof  that,  for 
any  pole  7;J  ,  if  the  symbolic  expression  for  &juc(k  ()  is  formally  different  from  zero  then  for  all  values  of 
k  <K(j  ,/ ),  a  jit,  is  also  formally  different  than  zero. 

If  L  (/)=!,  then  P,(t)  is  of  the  form  atk  e1*  .  This  corresponds  to  the  case  where  there  is  only  one 
directed  path  from  the  original  state  to  state  j  and  all  the  transition  rates  along  this  path  are  all  equal, 
i.e.  gy  =-7.  Only  in  the  L  (j )  =  1  case  is  there  a  chance  of  having  a  symbolic  expression  formally  equal 
to  zero.  Thus  P,  (f )  may  be  written 


iGJ{i)  l -l  0 


*-0 


(4) 


But,  especially  when  the  Pj(t)’s  have  common  poles,  i.e.  when  |r(/(t'))|  <  XI  I  r(j  )  |  ,  this 


expression  reduces  to 


t 

Pi( 0=  E  / 

7er(y(<))  0 


E  ¥ 

fe/(0  (-1 


hi,,  -1)  |  <iji*jikzk 


»  *,(i> 

-  E  /  E  a,.kxk^t^Ux 

lGT(J(i))  0  »-0 

where 


=  E  IL!  f  (7,(  “  0 ji*  >  )) 

i  e  /(•)  1-1  J 

and 

*Q(-r)=  max  (  K(j,r)  |  r=l . L(j),  and  7,v=7}. 

Moving  the  integral  inside  the  summation  we  obtain 


(5) 

(6) 

(7) 


*,b)  ‘ 

p,( 0=  E  E  xkt^‘>dz  . 

ier(j(<))  *-0  0 

The  resolution  of  this  integral  depends  on  whether  7  differs  from  7*  .  If  7=7* ,  then, 


(8) 


S  f 

f  x*e^*>*dx  =  /  z*<f*  =  "r~ 


*  +  1 


(*+l)  ‘ 


Otherwise,  when  a=(7-7*),  integration  by  parts  yields 


(9) 


,1 


We  can  now  write  the  equation  for  Pi  (f ) 


p,(0  = 


K.W 

/ 

Wi-i  £o 

«*•'**  [ 

+  ~ai.k (-i)*+1  .*•]  +  W,'e  ’ ' 

a  k-  0  (*+] 


We  note  that 


and  that 


r(0  =  h'}ur(;(«-)) 


if  la  i* 

K(i,l)=  Ki(‘ia)+ 1  if  Tffl=7»  and  Tf*  er(/(i)) 
0  if  7a  =7*  and  7*  £T(J(i )) 

From  (11)  note  that,  if  a  —  k-l ,  we  have 


K'b)  r*  ,*.< 

£  ®%*  Ef"1)1  /.  X  |+i 

*  —o  i-o  a 


r  *  k  i 

-S’-*. 

*— 0  L«-o  u !  Or 

-  S  <•  E  77  X  -rnr; 

.-0  *-«  tt!  o*+1  * 


*.h)  *.H  , 

-  E  <*  E  x  -Jjj. 

*  — o  r— *  *  !  a  * 


If  we  write  P,(t)  as 


for  all  75^7*  ,  we  have 


MO  _  .  Kti,i) 

p<(0=  £«v  £ 


I-l  k—0 


n  * 

a  ■ <l.k 1 


I  =  1,... 


1  171  I  t 

=  £  Mr*  Kr  ~  (7_7<r-*+1 


*  =  o . AT, (7) 


and  for  7* 


7 


4ii)  *.(■»/)  a,,* 

-  S  (-»—*< -3 


(•7 a -Of*) 


£  0  Tr,/ .0 


Moreover,  if  7*  £r(J(i )), 


(*+U 


*=0 . 


With  these  equations,  we  can  easily  compute  the  coefficients  of  the  polynomials  in  t  that  multiply  the 
exponentials  in  the  state  probability  expressions. 

If  we  associate  with  each  state  i  a  reward  level  r,- ,  then  the  exp  ted  instantaneous  reward  rate  for 
the  process  at  time  t  is  given  by  y)r.-  P,  (t ).  Its  polynomial  expression  can  be  easily  obtained  from  the 

I 

polynomial  expressions  of  the  ”P,  (/)”’s.  If  we  consider  the  system  to  be  up  in  all  non-absorbing  states, 
then  we  assign  r,-  =  1  to  all  such  states  and  r,-  =  0  to  all  absorbing  states.  In  such  a  case,  the  evaluation 
of  will  give  the  system  reliability  /?(<)•  On  the  other  hand,  if  some  of  the  non-absorbing 

states  are  system  down  states  which  are  assigned  reward  0,  then  the  evaluation  of  )  will  give  the 

point  (instantaneous)  availability  A  (t )  of  the  system.  In  the  general  case,  we  are  not  restricted  to  binary 
values  of  the  reward  rate  and  we  can  allow  several  (degraded)  levels  of  functionality. 


2.2  Implementation 

In  this  section  we  discuss  the  procedure  used  by  the  ACE  program  to  compute  state  probability 
expressions  for  an  acyclic  Markov  Chain.  We  first  note  that  equation  (15)  can  be  re-written  in  a  recursive 
fashion  to  greatly  reduce  the  number  of  operations  needed  to  compute  the  coefficients  of  the  state 
probability  expressions.  In  place  of  equation  (15),  we  can  write 

,  ®  7,/C,  (r)  j , 

a7./C,(7)  =  — -  ^ 


fl7  ,*  = 


(« 7.*  -  (*+1)  a7.*+i  ) 


k  =  A',('?)-1,..,0 


The  equations  for  computing  the  coefficients  of  the  outgoing  poles  (16  and  17)  remain  unchanged. 
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When  the  procedure  begins,  the  states  are  sorted  according  to  the  partial  order  induced  by  the 
transitions.  For  state  t ,  the  probability  expression  is  computed  by  first  determining  all  the  poles  of  states 
that  have  transitions  leading  to  state  »' .  Coefficients  for  the  polynomials  multiplying  the  incoming 
exponential  terms  are  computed  using  equations  (18)  and  (19).  If  the  pole  associated  with  state  i ’s 
outgoing  transition  rate  is  not  in  the  incoming  set,  the  new  pole’s  polynomial  multiplier  is  a  constant 
computed  using  equation  (16).  If  the  outgoing  pole  is  also  in  the  set  of  incoming  poles,  the  degree  of  its 
polynomial  will  be  incremented.  The  new  coefficients  for  the  incremented  polynomial  can  be  computed 
using  equation  (16)  and  (17). 


We  comment  here  on  the  execution  speed  of  this  procedure.  We  first  observe  that,  in  a  partially 
ordered  chain,  state  i  has  at  most  t-1  ancestors.  The  number  of  coefficients  in  a  state  probability 
expression  is  bounded  above  by  the  number  of  ancestors  the  state  has.  Therefore,  to  compute  the  state 
probability  distributions  of  all  the  states  in  an  n-state  acyclic  Markov  chain  we  need  to  compute  at  most 

^  +  coefficients.  The  procedure  described  above  requires  at  most  2  floating  point  multiplies  and/or 

divides  per  coefficient  computed.  Thus  the  number  of  multiplications  required  to  compute  all  the  state 
probability  expressions  for  an  acyclic  chain  is  bounded  by  (n  +l)*n  multiplies.  The  number  of  floating 
point  additions  for  state  i  is  bounded  by  «-l.  So  even  if  additions  are  included  in  the  operations  count, 
the  total  number  of  floating  point  operations  is  bounded  by  1.5  n*(n+l).  In  contrast,  a  typical  ODE 
solution  package  will  require  about  20*n3  floating  multiplies  to  compute  that  transient  solution  of  an 
arbitrary  CTMC  for  a  single  point  (or  small  set  of  grid  points).22  Other  methods  that  do  not  take  into 
account  the  special  structure  of  acyclic  CTMC  have  comparable  runtimes.  Even  if  other  numerical 
methods  are  adapted  to  specifically  solve  acyclic  chains,  the  ACE  algorithm  is  still  competitive. 


The  first  version  of  ACE  supports  an  unlimited  number  of  symbolic  variables  as  input  but  generates 
answers  that  are  symbolic  only  in  t .  The  coefficients  of  the  polynomials  in  t  that  multiply  the 
exponentials  are  numeric.  The  second  version  of  ACE,  which  is  currently  being  implemented,  will  be  fully 
symbolic  in  one  variable  and  numeric  in  other  variables,  i.e.  the  coefficients  of  the  polynomials  in  t  that 
multiply  the  exponentials  have  both  numeric  and  symbolic  parts.  This  allows  us  to  conduct  a  parametric 
sensitivity  analysis  in  a  fully  symbolic  fashion,  without  repeated  computation. 


3.  Sensitivity  Analysis 


Ii  is  often  of  interest  to  the  design  engineer  to  determine  the  performance  or  reliability  "bottleneck” 
of  the  system.8,  23  Towards  this  end,  it  is  desirable  to  evaluate  the  derivative  of  the  desired  measure  with 
respect  to  various  significant  system  parameters.  The  parameter  with  the  largest  derivative  deserves  the 
designer’s  interest  in  the  quest  to  improve  the  characteristics  of  the  designed  system.  Such  derivatives 
could  also  be  used  in  an  overall  system  optimization  effort  based  on  gradient  search  techniques.  We 
discuss  such  automated  sensitivity  analysis  next. 


3.1  Basic  Approach 


We  assume  that  some  transition  rates  qi}-  are  functions  of  some  parameter  X.  We  seek  a  symbolic 

dPf(t) 

expression  in  t  for  the  derivative  — — —  for  a  given  value  of  X. 

a  X 


Let: 

P  be  the  row  vector  of  the  P,  (t  )’s, 

Q  be  the  previously  defined  triangular  infinitesimal  generator  matrix  of  the  acyclic  CTMC, 

dPi  (t ) 

S  be  the  row  vector  of  the  sensitivities  — — — , 

0  A 

P(respectively  S)  be  the  vector  of  the  derivative  with  respect  to  time  t  of  P(respectively  S). 
Then,  from 


P  -PQ, 


(20) 


we  obtain 


S  =  SQ  +  PV 


(21) 


.  .  U I 

where  matrix  V  is  the  derivative  of  Q  with  respect  to  X,  i.e.  V r. ( t>,;-  }  such  that  vi}-  — —  .  So  S  is  the 

d  A 


solution  of  the  first  order  differential  system  (21). 

Defining  |  1  </  <N  ,  ,  and  }  and  taking  into  account  the  triangularity  of  Q, 

we  have 


Si  (t)  =  £  Sy  (l)  +  q„-  S,(t )  +  Y]  vfi  Pj(t)  +  v,v  P,  (t 


(22) 


T 


M0"9*M0  =  Wi(t] 


Wi(t)±  £  «*M0  +  £  vjip}(t)  +  v„p,(t) . 

*'€/(•)  /€i(i) 

Its  solution  is  of  the  form  24 


5,  (t)=  ‘  /  W,-  (z  )<  *"  *  dx  +  C,-  e  "*» ‘  .  (25) 

0 

Because  because  for  all  t  ,  Pt  (0)  is  a  constant,  we  have  C,  =  0  for  all  i .  Using  the  notation  7*  for  -qi{ , 
we  then  get 

t  t  t 

MO  =  £  +vilfpiw',-''dz  .  (26) 

yeJ(f)  0  iel(i)  0  0 

In  initial  state  t ,  /(« )  =  l(i)  =  <j>  and  S,-(t)  has  the  following  form: 


MO  =  ®«/p(*)<1r*(‘"*)<fe 


=  v,v  f,,‘  •  I 

It  is  easily  seen  that,  for  any  state  »' ,  « =  1,..,JV ,  5,-  (t )  may  be  written  as  a  polynomial  of  the  form 

M0-£«v  [£»„*<*  , 

Moreover,  the  set  of  poles  corresponding  to  5,  (<  )  is  included  in  r(«  ). 

To  get  the  recursive  algorithm  for  computing  the  polynomial  form  of  S,-  (t ),  we  first  assume  that 


(_l  *- 0 


.  yei(0 


where,  by  convention  K(j  ,1)  =  -1  if  there  is  no  exponential  term  with  respect  to  7 )t  while  7;j  Gr(;). 
Then,  using  equation  (26),  we  show  that  S,-(t)  may  be  also  written  in  the  same  form  and  we  determine 
the  expression  of  the  new  K  (i  , /  )’s  and  bM ’s: 


5,(0=  E  l/iSl*1’'’  f/C£,)6^x*le''*(‘-*)(fx 


;£/(,')  1-10  I  *— 0 


i-io  [*=o 
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/  =1  0  I  <fc=0 


x*  e^-^x  . 


Let  us  introduce  the  following  definitions 


K  h)  =  max  \x{i,r  )  |  r=l,..L(y)  and  Try,  =nr )  (31a) 

Kih)  =  I  r=l, ..!(;■)  and  7,>=t|  •  (31b) 

By  convention,  we  take  K,(' 7)  = -1,  (respectively  i£,(7)  =  -l),  if  the  set  in  the  definition  (31a) 
(respectively  (31b))  is  empty.  Then  from  relation  (30)  and  taking  into  account  the  influence  of  the 
convolution  operation  over  the  power  of  / ,  we  get: 


For 


K  (« ,/ )  ^  max  |  K(  (7)  ,  &  (7)  ,  (-!(.„  -0)  +  1(,„  ^0)^  («  J  ))  j 


Note  that  K  (i  ,7)  =  -1  iff  v,y  =  0  and  (7)  =  Ki  (7)  ==  -1. 


For  7* 


K  (•  ,7* )  =  max |  (7 * )  +  l(/ri(7.)>0)  ,  Ki{l)  +  !(*,(,.)><>)  »  (-!(,„ -0)  +  1(,M  *o)[K{i 7*  )+l))  |  .  (33) 


On  the  other  hand,  we  also  get  from  (30) 


5,(0=  E  E  e‘’“i%tfxke<”*)dx 

7£r{J(i)k  *=0  0 


^  i ,*  =  ^(7? ,b)>o)  E  1j<  Vi .1  ="r)0ik 

;€/(,•)  (=i 


(i)>0)  E  v;*  E  f  (t;i  =7)°;* 


ye^(,)  (=-1 


+  v,v  o  7l* 


Vk=ol...ki(7) ,  Vrer(i) 


a  |  *<■'") 


^i(7)  =  \  K(i, 7*)-l 


if  77^7* 
otherwise 
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Note  that  6 -,»,<)  =  a'  if  'I*  £r(/(i )).  The  form  of  the  relation  (34)  is  equivalent  to  the  one  of 

relation  (5)  up  to  the  last  term  of  (34).  Therefore  we  get  similar  recursive  relations  for  the  computation  of 
the  b  coefficients. 


4.  Cumulative  Measures 

Thus  far,  we  have  discussed  instantaneous  measures  associated  with  a  CTMC.  There  are  many 
cumulative  measures  of  interest  as  well.  The  most  common  such  measure  is  the  mean  time  to  failure,  We 
first  discuss  cumulative  measures  for  a  Markov  reward  process  and  then  specialize  to  various  cases.14  Let 
Z(t )  be  the  state  of  the  CTMC  at  time  t .  Define  the  accumulated  reward  in  the  interval  [0,1  ]  as9' 11 

t 

frz^dr.  (39) 

0 

Then  the  expected  cumulative  reward  in  the  interv*.  [0,t]  is  given  by: 

^IV(I))  -  £  JriPi(x)dx  (40) 

i—l  0 


If 
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A  N  W 

~C  1,k  =  2  ri  £  »od  *  </C(i  ,1 


i-1  (-1 


if  (t)  =  maximum  power  of  t  with  respect  to  pole  7  over  all  states  with  r,-  ^0 


5  [HO]  =  Ei'eT  ,kfzke1*dz 


■j  *  — 0  0 

Note  that  a  pole  7=0  had  been  introduced  by  all  absorbing  states  and  consequently 

to*  =0  for  k  >0  . 

Moreover,  if  the  reward  is  null  for  all  the  absorbing  states,  c  00  =  0.  After  the  integration  we  get 

E  [^(01  =  S  ei.*  *  *  e  11  +  eo,o  +  c0ilf 
7/0  k  -0 

where 


f-il  = 


Ccxk  (*  +1)  C7.*+l)  ,  „  TSt  \  „ 

c7l*  ==  - - -  k=0,..,K  (7)-l  ,  V7^0 


c  0,0 - XI  C  1,0  c  0,1  =  C  0,0  • 

7/0 


We  have  the  following  special  cases: 


{0  if  t  is  a 
1  otherwi 


if  «  is  an  absorbing  state 
otherwise 


•  lim2?[T(<)]  =  c0, 0  is  equivalent  to  the  "mean  time  to  absorption”  (MTTA). 

t  — »00  ’ 

Note  that  in  this  case  e0,i  =  0 


j  0  if  *  £  set  of 
^  r‘  1 1  if  t  £  set  of 


down  states 
up  states 


Then: 


where  the  absorbing  states  must  belong  to  the  set  of  down  states. 


Urn  E[V(i)]  is  equivalent  to  the  ”mean  up  time  before  absorption”  (MUTBA). 


•  E[Y(t )]  is  the  "expected  up  time”  on  the  interval  [0 ]. 

•  —  ^  is  the  "expected  interval  availability”  on  [0,<]. 

The  sensitivity  of  E  [T(f  )J  can  also  be  derived.  If  we  define  A  to  be  the  set  of  numerical  values  of  X 
which  would  make  two  formally  different  poles  numerically  identical,  then  we  can  write 

N  * 

=  S  Jst  {x  )dx  (45) 

■  — l  o 

Here  the  differentiation  under  the  integral  may  be  taken  because,  for  all  X  £  A,  the  integral  exists  and 
the  derivative  with  respect  to  X  is  continuous.25 


Let  K (of)  =  the  maximum  power  of  l  with  respect  to  pole  7  in  the  S,  (t)  expressions  over  all  states  1  such 
that  r,-  f^O.  Let 


A  N  W 

=  S  r<  2j  ^ 


•-1  (-1 


m,  k  —Q,..,K  (7) 


0  0 


The  integration  is  quite  similar  to  the  one  above  of  E  [J”(f )].  We  get 

1*0  k-0 

where 


d%Kh)  ~ 


Ui,*  ~  (*  +1)  drk  +1) 


k  =  K(- r)-l,..,0  ,  Vt^O 


^0,0 - ^i.o  >  ^0,1  =  do.o  ■ 

1*0 


Note  that,  if  the  absorbing  state  is  unique,  or  if  the  reward  is  null  for  all  the  absorbing  states,  then 
d nn  =  0  and  therefore  dm  =0. 


5.  Symbolic  and  Numerical  Examples 

In  this  section,  we  demonstrate  the  use  of  ACE  state  probability  computation,  sensitivity  analysis, 
and  expected  reward  computation.  Our  first  example  is  the  solution  and  sensitivity  analysis  of  a 
medium-sized  acyclic  CTMC  that  is  used  in  the  analysis  of  a  stochastic  PERT  network.26  We  also 
determine  the  sensitivity  of  the  solution  with  respect  to  one  of  the  transition  rates. 

Our  second  example  considers  the  fault-tolerant  computer  system  model  given  in  Figure  1.  First  we 
solve  for  the  reliability  of  the  original  model.  We  compare  the  exact  solution  to  the  reliability  estimate 
obtained  by  using  two  approximate  models.  Assigning  rewards  to  various  states  of  the  model,  we  then 
compute  cumulative  measures  of  the  system’s  performance  and  analyze  the  sensitivity  of  these 
performance  estimates  with  respect  to  the  system  parameters. 

5.1  Solution  and  Sensitivity  Analysis  of  a  Stochastic  PERT  Network 

We  consider  the  analysis  of  a  stochastic  PERT  network.26  The  chain  that  must  be  solved  to  analyze 
the  network  is  given  in  Figure  2a.  The  chain  has  24  states  and  its  solution  is  clearly  too  complex  for 
practical  hand  computation  or  a  general  purpose  symbolic  solver.  In  stochastic  PERT  analysis,  we  are 
interested  in  the  distribution  of  the  time  needed  to  reach  the  absorbing  state.  For  an  absorbing  state,  the 
probability  that  the  process  reaches  the  state  on  or  before  time  t  is  given  by  the  probability  of  being  in 
the  state  at  time  t .  Using  the  parameter  values  given  in  Figure  2b,  we  use  the  numerical  ACE  algorithm 
to  find  the  probability  of  being  in  the  absorbing  state  (state  24)  as  a  function  of  time  t .  The  numerical 
expression  in  terms  of  t  is  given  in  Table  2. 

In  Table  3,  we  give  the  sensitivity  of  the  solution  with  respect  to  parameter 

5.2  Analysis  of  Exact  and  Approximate  Models  of  A  TMR  System 

We  consider  the  example  presented  in  Figure  1,  a  TMR  (Triple  Modular  Redundancy)  System  with 
two  cold  spares.  We  assume  that  the  failure  rate  of  an  active  unit  is  X  and  that  of  a  spare  unit  is  zero. 
Upon  occurrence  of  a  failure,  the  system  takes  an  exponentially  distributed  time  (with  parameter  S)  to 
switch  out  the  failed  unit.  Should  another  active  unit  fail  during  this  switching  process,  we  consider  the 
system  to  have  failed.  We  do  not  allow  any  repairs  of  failed  units.  The  state  diagram  of  this  system  is 


shown  in  Figure  1. 


We  note  that  the  reliability  of  the  system  at  time  t ,  denoted  R  (t ),  is  given  by  1  -PA  (t ).  Taking  X 
to  be  10“*  and  6  to  be  1.0,  we  use  the  ACE  package  to  derive  the  expression  for  PA  (t )  given  in  Table  4a. 

We  consider  an  approximate  model  of  the  system.  The  instantaneous  coverage  model,  replaces  the 
recovery  states  by  probabilistic  branch-points.  The  coverage  constant  c  denotes  the  probability  of 
successful  recovery  from  a  processor  failure.  In  this  example,  recovery-  is  unsuccessful  only  if  a  second 
processor  failure  occurs  during  the  recovery  period.  Thus  the  probability  of  successful  recovery  can  be 

computed  as  c  =  ^  ^  ,18  A  special  case  of  this  model,  the  perfect  coverage  model,  assumes  that 

recovery  is  always  successful.  In  this  case,  e  —  1.0.  The  aggregated  chain  for  the  approximate  model  is 
shown  in  Figure  3.  The  unreliability  expressions  for  the  instantaneous  coverage  and  perfect  coverage 
models  are  given  in  Table  4b  and  Table  4c.  In  Figure  4  we  plot  - log 1(j  of  the  system  unreliability  as  a 
function  of  time.  We  see  that  the  constant  coverage  model  provides  a  fairly  tight  lower  bound  on  the 
actual  solution.27  The  perfect  coverage  model  provides  a  less  tight  upper  bound. 

5.3  Computation  of  Cumulative  Measures  for  a  TMR  System 

We  consider  the  computation  of  cumulative  reward  measures  for  the  chain  analyzed  in  the  previous 
section.  It  is  natural  to  assign  reward  rate  one  to  states  (3,2),  (3,1),  (3,0),  and  (2,0)  and  reward  rate  zero 
to  the  absorbing  state  A.  If  the  switching  time  is  small,  we  may  assign  reward  rate  one  to  states  (2,2)  and 
(2,1)  if  the  system  requirements  allow  for  temporary  "down  time”.  On  the  other  hand,  we  may  assign 
reward  rate  0  to  these  two  states  if  we  are  interested  in  actual  system  "up  time”  in  the  given  observation 
period. 

Using  the  numeric  values  suggested  previously,  X  =  10-4  and  6  —  1.0,  we  evaluate  the  TMR 
system’s  mean  up  time  in  the  interval  [0,tj.  Using  the  reward  vector  with  reward  level  0  assigned  to  the 
recovery  states,  we  give  the  expression  for  the  expected  cumulative  reward  earned  up  to  time  t  in  Table 
5.  We  note  that  the  constant  term  (14,996)  in  the  expression  gives  expected  up  time  before  absorption 
(MUTBA).  In  Tables  6a  and  6b  respectively,  we  give  the  sensitivity  of  the  expected  reward  with  respect 
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6.  Conclusions 

Efficient  generation  of  exact  numeric  and  exact  symbolic  expressions  for  the  transient  behavior  of 
CTMC  should  provide  at  least  three  benefits.  First,  it  should  be  possible  to  compare  the  results  obtained 
by  approximate  solution  methods  for  small  to  medium  sized  CTMC  with  exact  solutions  for  the  same 
problems.  By  showing  the  magnitude  and  nature  of  the  error  that  approximate  solutions  introduce,  this 
type  of  analysis  should  provide  a  good  indication  of  an  aggregation/approximation  technique’s  utility  for 
larger,  more  realistic  problems.  Second,  symbolic  solutions  allow  us  to  easily  examine  the  influence  of 
changing  parameter  values  on  the  solutions  of  Markov  models.  This  type  of  investigation  could  be 
expensive  using  conventional  simulation  or  numerical  solution  techniques.  Symbolic  solutions  also  provide 
the  basis  for  using  Markov  models  in  mathematical  programs  for  design  optimization.  By  providing  exact 
and  exact  symbolic  solutions  of  CTMC,  the  ACE  package  should  enhance  our  ability  to  study  Markov 
reliability  models,  and  approximation  techniques  for  their  solution. 

Further  theoretical  efforts  include  the  efficient  closed-form  solution  of  cyclic  chains,  the  analysis  of 
restricted  semi-markov  processes,  and  reward  based  computations.  Practical  extensions  include 
interfacing  the  ACE  procedure  with  existing  modeling  tools  and  the  addition  of  a  block  definition  and 
solution  facility.  The  user  will  be  able  to  define  blocks  of  states  with  fixed  entry  and  exit  points.  The 
blocks  could  be  evaluated  by  direct  insertion  of  their  states  into  the  chain.  Alternatively,  the  block  could 
be  solved  in  isolation  and  replaced  by  approximately  equivalent  acyclic  sub-chains.  This  capability  should 
further  facilitate  the  use  of  ACE  in  evaluating  aggregation  methods. 


References 


1.  D.  F.  Towsley,  J.  C.  Browne,  and  K.  M.  Chandy,  “Models  for  Parallel  Processing  Within 
Programs:  Applications  to  CPU:I/0  and  I/0:I/0  Overlap,”  CACM,  pp.  821-31,  October 
1978. 

2.  Steven  A.  Lavenberg,  ed.,  Computer  Performance  Modeling  Handbook,  Academic  Press, 
1983. 

3.  Kishor  S.  Trivedi,  Probability  &  Statistics  with  Reliability,  Queuing  &  Computer  Science 
Applications,  Prentice-Hall,  1982. 

4.  Ying-Wah  Ng  and  Algirdas  Avizienis,  “A  Unified  Reliability  Model  for  Fault-Tolerant  Com¬ 
puters,”  IEEE  Transactions  on  Computers,  pp.  1002-1011,  November  1980. 

5.  Robert  Geist  and  Kishor  Trivedi,  “Ultra-High  Reliability  Prediction  for  Fault^Tolerant 
Computer  Systems,”  IEEE  Transactions  on  Computers,  pp.  1118-1127,  December,  1983. 

6.  A.  Costes,  J.  E.  Doucet,  C.  Landrault,  and  J.C.  Laprie,  “SURF:  A  Program  for  Dependabil¬ 
ity  Evaluation  of  Complex  Fault-Tolerant  Computing  Systems,”  in  Proceedings  IEEE  11-th 
Fault  Tolerant  Computing  Symposium,  pp.  72-78,  June  1981. 

7.  Daniel  P.  Siewiorek  and  Robert  S.  Swarz,  The  Theory  and  Practice  of  Reliable  System 
Design,  Digital  Press,  1982. 

8.  Ambuj  Goyal,  Steve  Lavenberg,  and  Kishor  Trivedi,  “Probabilistic  Modeling  of  Computer 
System  Availability,”  The  Conference  on  Statistical  and  Computational  Problems  in  Proba¬ 
bility  Modeling,  Annals  of  Operations  Research,,  1986. 

9.  John  F.  Meyer,  “On  Evaluating  the  Performability  of  Degradable  Computer  Systems,” 
IEEE  Transactions  on  Computers,  August  1980. 

10.  Ambuj  Goyal  and  Asser  Tantawi,  “Evaluation  of  Performability  in  Acyclic  Markov 
Chains,”  IBM  Computer  Science  Research  Report,  May  1984. 

11.  V.  Kulkarni,  V.  Nicola,  K.  Trivedi,  and  R.  Smith,  A  Unified  Model  for  the  Analysis  of  Job 
Completion  Time  and  Performability  Measures  in  Fault- Toler  ant  Systems,  submitted  for 
publication. 

12.  D.R.  Cox,  “The  Use  of  Complex  Probability  in  the  Theory  of  Stochastic  Processes,”  in 
Proceedings  Cambridge  Philosophical  Society,  1955. 

13.  Marcel  F.  Neuts,  Matrix  Geometric  Solutions  in  Stochastic  Models:  An  Algorithmic 
Approach,  John  Hopkins  University  Press,  1981. 

14.  Sheldon  M.  Ross,  Stochastic  Processes,  J.  Wiley  &  Sons,  1983. 

15.  J.  J.  Stiffler,  L.  A.  Bryant,  and  L.  Guccione,  “CARE  III  Final  Report  Phase  I,”  NASA  Con¬ 
tractor  Report  159122,  November  1979. 

16.  D.R.  Miller,  “Reliability  calculation  using  randomization  for  markovian  fault-tolerant  com¬ 
puting  systems,”  in  Proceedings  IEEE  18-th  Fault  Tolerant  Computing  Symposium,  pp.  284- 
289  ,  1983. 

17.  Andrea  Bobbio  and  Kishor  Trivedi,  “An  Aggregation  Technique  for  the  Transient  Analysis 
of  Stiff  Markov  Systems,”  IEEE  Transaction  on  Computers,  to  appear. 

18.  Kishor  Trivedi,  Robert  Geist,  Mark  Smotherman,  and  Joanne  Bechta  Dugan,  “Hybrid 
Modeling  of  Fault-Tolerant  Computer  Systems,”  International  Journal  of  Computers  and 
Electrical  Engineering,  vol.  11,  no.  2/3,  pp.  87-108,  1984. 

19.  Kishor  Trivedi  and  Robert  Geist,  “A  Tutorial  on  the  CARE  III  Approach  to  Reliability 
Modeling,”  NASA  Contractor  Report  3488,  1981. 

20.  K.  Shin  and  Y.  Lee,  “Error  Detection  Process  -  Model,  Design,  and  Its  Impact  on  Computer 
Performance,”  IEEE  Transactions  on  Computers,  vol.  C-33,  June  1984. 

21.  Robin  Sahner  and  Kishor  Trivedi,  “SPADE:  A  Tool  for  Performance  and  Reliability  Evalua¬ 
tion,”  International  Workshop  On  Techniques  and  Tools  for  Performance  Analysis,  Sophia 


Antipolis,  France,  June  1985. 

22.  C.  Moler  and  C.  Van  Loan,  “Nineteen  Dubious  Ways  to  Compute  the  Exponential  of  a 
Matrix,”  SIAM  Review,  vol.  20,  pp.  801-836,  Oct.  1978. 

23.  P.  Frank,  Introduction  to  System  Sensitivity,  Academic  Press,  1978. 

24.  R.  Buck  and  E.  Buck,  Introduction  to  Differential  Equations,  Houghton  Mifflin,  1976. 

25.  W.  Rudin,  Principles  of  Mathematical  Analysis,  McGraw-Hill,  1976. 

26.  V.  Kulkarni  and  V.  Adlakha,  “Markov  and  Markov-Regenerative  PERT  Networks,”  Opera¬ 
tions  Research,  to  appear. 

27.  John  McGough,  Mark  Smotherman,  and  Kishor  Trivedi,  “The  Conservativeness  of  Reliabil¬ 
ity  Estimates  Based  on  Instantaneous  Coverage,”  IEEE  Transactions  on  Computers,  vol.  7, 
pp.  602-609,  July  1985. 

28.  R.  B.  Conn,  P.  M.  Merryman,  and  K.  L.  Whitelaw,  “CAST  -  A  Complimentary  Analytic- 
Simulative  Technique  for  Modeling  Fault-Tolerant  Computing  Systems,”  in  Proceedings 
AIAA  Computers  in  Aerospace  Conference,  Los  Angeles,  November,  1977. 

29.  S.V.  Makam  and  A.  Avizienis,  “ARIES  81:  A  reliability  and  life-cycle  evaluation  tool  for 
fault-tolerant  systems,”  in  Proceedings  IEEE  12-th  Fault  Tolerant  Computing  Symposium, 
pp.  267-274,  June  1982. 


m 


v  V  V  * 

1  .  '  Jl  "  -  ■  " 


m 


Figure  2  -  GTMC  for  Stochastic  PERT  Network 


Method:  Simulation 

Package:  CAST  [28] 

Note:  Allows  full  simulation  of  a  restricted  class  of  systems 

Method:  Differential  Equations  (Numerical  Solution) 

Package:  HARP  [18] 

Domain:  Homogeneous  (cyclic)  and  (acyclic)  Non-Homogeneous  CTMC 

Method:  Differential  Equations  (Laplace  Solution) 

Package:  SURF  [6] 

Domain:  Non-Markov  Processes 

Note:  Approximate  solution  using  Coxian  method  of  stages 

Method:  Integral  Equations  (Numerical  Solution) 

Package:  Care  III  Coverage  Model  [15] 

Domain:  Semi-Markov  Processes 

Package:  Care  III  Reliability  Model  [15] 

Domain:  Non-Homogeneous  acyclic  CTMC 

Method:  Uniformization 

Package:  SAVE  [8] 

Domain:  Homogeneous  Markov  Processes  with  Rewards 

Method:  Eigenvalue  Decomposition 

Package:  ARIES  [29] 

Domain:  Cyclic  Homogeneous  CTMC 

Note:  Poles  and  their  coefficients  derived  numerically 

Method:  Closed  Form  Solution 

Package:  ARIES  [2Sj 

Domain:  Acyclic  Homogeneous  CTMC 
Note:  Distinct  Eigenvalues  Required 

Package:  ACE 

Domain:  Acyclic  Homogeneous  CTMC 
Note:  State  probability  expression  derived  symbolically  in  t 
No  restriction  on  eigenvalues 
Other  symbolic  variables  optionally  available 
Sensitivity,  cumulative,  and  reward-based  measures 


Table  1  -  Reliability  Modeling  Packages  Employing  Markov  Chain  Techniques 


+  (-6.58467)  *  (t  **  0)  *  exp  (-2.7  t) 
+  (-0.604801)  *  (t  **  0)  *  exp  (-3  t) 
+  (-58.6153)  *  (t  **  0)  *  exp  (-1.5  t) 
+  (10.3929)  *  (t  **  1)  *  exp  (-1.5  t) 

+  (0.959701)  *  (t  **  0)  *  exp  (-1.1  t) 

+  (-44.0977)  *  (t  **  1)  *  exp  (-1.1  t) 

+  (412.962)  *  (t  **  0)  *  exp  (-1.7  t) 

+  (-279.099)  *  (t  **  0)  *  exp  (-1.8  t) 

+  (-200.911)  *  (t  **  0)  *  exp  (-1.8  t) 

+  (67.2981)  *  (t  **  0)  *  exp  (-0.6  t) 
+  (-5.71429)  *  (t  **  1)  *  exp  (-0.6  t) 

+  (-236.801)  *  (t  **  0)  *  exp  (-1.2  t) 

+  (1.94164)  *  (t  **  0)  *  exp  (-1.3  t) 
+  (190.687)  *  (t  **  0)  *  exp  (-1  t) 

+  (39.8632)  *  (t  **  0)  *  exp  (-1.6  t) 
+  (-34.149)  *  (t  **  0)  *  exp  (-0.8  t) 

+  (-22.2647)  *  (t  **  0)  *  exp  (-0.5  t) 
+  (48.6971)  *  (t  **  0)  *  exp  (-2.1  t) 
+  (75.1147)  *  (t  **  0)  *  exp  (-2  t) 

+  (0.506549)  *  (t  **  0)  *  exp  (-3.5  t) 

+  (1) 


Table  2  -  State  Probability  Expression  For  PERT  Chain  Absorbing  State 


+  (46.6468)  *  (t  **  0)  *  exp  (-2.7  t) 

+  (6.58467)  *  (t  **  1)  *  exp  (-2.7  t) 

+  (-13.5139)  *  (t  **  0)  *  exp  (-3  t) 

+  (1.2096)  *  (t  **  1)  *  exp  (-3  t) 

+  (853.74)  *  (t  **  0)  *  exp  (-1.5  t) 

+  (-923.911)  *  (t  **  0)  *  exp  (-1.1  t) 

+  (-1.11422)  *  (t  **  1)  *  exp  (-1.1  t) 

+  (1655.42)  *  (t  **  0)  *  exp  (-1.7  t) 
+  (-412.962)  *  (t  **  1)  *  exp  (-1.7  t) 
+  (1155.72)  *  (t  **  0)  *  exp  (-1.8  t) 

+  (279.099)  *  (t  **  1)  *  exp  (-1.8  t) 

+  (-2790.43)  *  (t  **  0)  *  exp  (-1.8  t) 
+  (5.27288)  *  (t  **  0)  *  exp  (-0.6  t) 
+  (-714.788)  *  (t  **  0)  *  exp  (-1.2  t) 
+  (236.801)  *  (t  **  1)  *  exp  (-1.2  t) 

4-  (1828.72)  *  (t  **  0)  *  exp  (-1.3  t) 

+  (-3.20483)  *  (t  **  0)  *  exp  (-1  t) 

+  (-1355.52)  *  (t  **  0)  *  exp  (-1.6  t) 

+  (-17.1684)  *  (t  **  0)  *  exp  (-0.8  t) 

+  (1.04781)  >•  (t  **  0)  *  exp  (-0.5  t) 

+  (26.3408)  *  (t  **  0)  *  exp  (-2.1  t) 

+  (242.907)  *  (t  **  0)  *  exp  (-2  t) 

+  (2.72702)  *  (t  **  0)  *  exp  (-3.5  t) 

+  (0) 


Table  3  -  Numerical  Sensitivity  Expression  For  the  Absorbing  State,  Parameter  /i2 


+  (26)  *  (t  **  0)  *  exp  (-0.0003  t) 

4-  (0.00240006)  *  (t  **  1)  *  exp  (-0.0003  t) 
4-  (9.0018e-08)  *  (t  **  2)  *  exp  (-0.0003  t) 

+  (5.994e-08)  *  (t  **  0)  *  exp  (-1.0002  t) 

+  (-1.80054e-ll)  *  (t  **  1)  *  exp  (-1.0002  t) 
4-  (-27)  *  (t  **  0)  *  exp  (-0.0002  t) 

+  (1) 


Table  4a  -  Unreliability  Expression  for  TMR  System 


4-  (25.9892)  *  (t  **  0)  *  exp  (-0.0003  t) 

4-  (0.00239898)  *  (t  **  1)  *  exp  (-0.0003  t) 
+  (8.9964e-08)  *  (t  **  2)  *  exp  (-0.0003  t) 
4-  (-26.9892)  *  (t  **  0)  *  exp  (-0.0002  t) 

+  (1) 


Table  4b  -  TMR  Unreliability  Expression  Instantaneous  Coverage  Approximation 


+  (26)  *  (t  **  0)  *  exp  (-0.0003  t) 

+  (0.0024)  *  (t  **  1)  *  exp  (-0.0003  t) 
+  (9e-08)  *  (t  **  2)  *  exp  (-0.0003  t) 
+  (-27)  *  (t  **  0)  *  exp  (-0.0002  t) 

+  (D 


Table  4c  -  TMR  Unreliability  Expression  Perfect  Coverage  Approximation 


+  (120004)  *  (t  **  0)  *  exp(  -0.0003  t) 

4-  (10.0009)  *  (t  **  1)  *  exp(  -0.0003  t) 

+  (0.00030006)  *  (t  **  2)  *  exp(  -0.0003  t) 
+  (-0.00029964)  *  (t  **  0)  *  exp(  -1.0002  t) 
+  (8.9982e-08)  *  (t  **  l)  *  exp(  -1.0002  t) 
+  (-135000)  *  (t  **  0)  *  exp(  -0.0002  t) 

+  (14996)  *  (t  **  0)  *  exp(  0  t) 


Table  5  -  Mean  Up  Time  in  [0,t]  for  TMR  System 


+  (-1.1995e+09)  *  (t  **  0)  *  exp  (-0.0003  t) 
-I-  (-359961)  *  (t  **  1)  *  exp  (-0.0003  t) 

+  (-27.0003)  *  (t  **  2)  *  exp  (-0.0003  t) 

+  (-0.00090018)  *  (t  **  3)  *  exp  (-0.0003  t) 
+  (-2.9946)  *  (t  **  0)  *  exp  (-1.0002  t) 

+  (0.00180018)  *  (t  **  1)  *  exp  (-1.0002  t) 

+  (1.34946e+09)  *  (t  **  0)  *  exp  (-0.0002  t) 
+  (270000)  *  (t  **  1)  *  exp  (-0.0002  t) 

+  (-1.4996e+08)  *  (t  **  0)  *  exp  (0  t) 


Table  6a  -  Numerical  Sensitivity  Expression  for  E[Y(t)]  with  respect  to  X 


+  (-3.99833)  *  (t  **  0)  *  exp  (-0.0003  t) 

+  (-0.00089992)  *  (t  **  1)  *  exp  (-0.0003  t) 

+  (-6.0018e-08)  *  (t  **  2)  *  exp  (-0.0003  t) 

+  (0.00059892)  *  (t  **  0)  *  exp  (-1.0002  t) 

+  (0.00029946)  *  (t  **  1)  *  exp  (-1.0002  t) 

+  (-8.9982e-08)  *  (t  **  2)  *  exp  (-1.0002  t) 

+  (-2.18834e-13)  *  (t  **  0)  *  exp  (-0.0002  t) 
+  (3.99773)  *  (t  **  0)  *  exp  (0  t) 


Table  6b  -  Numerical  Sensitivity  Expression  for  E[Y(t)]  with  respect  to  6 
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